Papers with post-training of large language models
InfoPO: On Mutual Information Maximization for Large Language Model Alignment (2025.naacl-long)
Copied to clipboard
Teng Xiao, Zhen Ge, Sujay Sanghavi, Tian Wang, Julian Katz-Samuels, Marc Versage, Qingjun Cui, Trishul Chilimbi
| Challenge: | Recent studies have shown that direct preference optimization and its variants can be useful for fine-tuning large language models with human preferences data. |
| Approach: | They propose a preference fine-tuning algorithm that effectively and efficiently aligns large language models using preference data. |
| Outcome: | Extensive experiments show that the proposed algorithm outperforms established baselines on reasoning tasks. |
How Instruction and Reasoning Data shape Post-Training: Data Quality through the Lens of Layer-wise Gradients (2026.acl-long)
Copied to clipboard
| Challenge: | Spectral properties of low/high-quality instruction and reasoning data are used to explain finetuning dynamics in large language models. |
| Approach: | They propose to analyze layer-wise gradients induced by low/high-quality instruction and reasoning data for LLM post-training. |
| Outcome: | The results show that higher-quality data are associated with lower nuclear norms and higher effective ranks. |